An Extended Variational Principle for the SK Spin-Glass Model 
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The recent proof by F. Guerra that the Parisi ansatz provides a lower bound on the free energy 
of the SK spin-glass model could have been taken as offering some support to the validity of the 
purported solution. In this work we present a broader variational principle, in which the lower 
bound, as well as the actual value, are expressed through an optimization procedure for which 
ultrametic/hierarchal structures form only a subset of the variational class. The validity of Parisi's 
ansatz for the SK model is still in question. The new variational principle may be of help in critical 
review of the issue. 

PACS numbers: 75.50.Lk 
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Introduction The statistical mechanics of spin-glass 
models is characterized by the existence of a diverse col- 
lection of competing states, very slow relaxation of the 
quenched dynamics, and a rather involved picture of the 
equilibrium state. 

A great deal of insight on the subject has been pro- 
duced through the study of the Sherrington Kirkpatrick 
(SK) model [1]. After some initial attempts, a solution 
was proposed by G. Parisi which has the requisite sta- 
bility and many other attractive features [2]. Its de- 
velopment has yielded a plethora of applications of the 
method, in which a key structural assumption is a par- 
ticular form of the replica symmetry- breaking (i.e., the 
assumption of "ultramctricity" , or the hierarchal struc- 
ture, of the overlaps among the observed spin configura- 
tions) [3]. 

Yet to this day it was not established that this very 
appealing proposal does indeed provide the equilibrium 
structure of the SK model. A recent breakthrough is the 
proof by F. Guerra [4] that the free energy provided by 
Parisi's purported solution is a rigorous lower bound for 
the SK free energy. 

More completely, the result of Guerra is that for any 
value of the order parameter, which within the assumed 
ansatz is a function, the Parisi functional provides a rig- 
orous lower bound. Thus, this relation is also valid for 
the maximizer which yields the Parisi solution. 

In this work we present a variational principle for the 
free energy of the SK model which makes no use of a 
Parisi-type order parameter, and which yields the result 
of Guerra as a particular implication. More explicitly, 
the new principle allows more varied bounds on the free 
energy, for which there is no need to assume a hierar- 
chal organization of the Gibbs state (e.g., as expressed in 
the assumed ultrametricity of the overlaps [3] ) . Guerra's 
results follow when the variational principle is tested 
against the Derrida-Ruelle hierarchal probability cascade 
models (GREM) [5]. 

This leads us to a question which is not new: is the 
ultrametricity an inherent structue of the SK mean-field 
model, or is it only a simplifiying assumption. The new 



variational principle may provide a tool for challenging 
tests of this issue. 

The model The SK model concerns Ising-type spins, 
a = (ci, . . . , oat), with an a-priori equi-distribution over 
the values {±1}, and the random Hamiltonian 
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where {Jij} are independent normal Gaussian variables. 

Our analysis applies to a more general class of Hamil- 
tonians which includes all the even "p-spin" models [6, 7]. 
Namely: 
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where all the {Ji lt ...,i r } are independent normal Gaussian 
variables (for convenience, the tensor is not assumed here 



to be symmetric), and S r =i 



= 1. As 



[7], 



our ar- 



gument requires that the function f(q) = X]^=i l a ?-| 2 Q r 
be convex on [—1,1]. 

One may note that Kn(o~) form a family of centered 
Gaussian variables with the covariance 
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which depends on the spin-spin overlap: q<r,a' — 
J2j a j a j- The standard SK model corresponds to 

/('/) T- 

The partition function, Z , the quenched free energy, F, 
and what we shall call here the pressure, P, are defined 

as 
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where E (— ) is an average over the random couplings 
{Jij}- The thermodynamic limit for the free energy, 
i.e., the existence of limjv^oo Pn(0, h) = P(/3,h), was 
recently established by Guerra-Toninelli [8] through a 
much-awaited argument. 

The Variational Principle Our variational expression 
for P(0, h) employs a setup which may at first appear 
strange, but is natural from the cavity perspective, when 
one considers the change in the total free energy caused 
by the addition of M spins to a much larger system of 
size N. The expression for Zn+mIZn simplifies in the 
limit N — > oo, at fixed M. In the following idealized 
definition one may regard the symbol a as representing 
the configuration of the bulk. The discretness seen in the 
definition (Yl £a) is just for the convenience of the formu- 
lation of the variational bounds, and not an assumption 
on the Gibbs state, though such an assumption may well 
be true. (A more general formulation is possible, but 
not much is lost by restricting attention to the "ROSt" 
defined below.) 

Definition (Random Overlap Structures): A ran- 
dom overlap structure (ROSt) consists of a probability 
space {O, fj,(du>)} where for each oj there is associated 
a system of weights {£«(w)} and an "overlap kernel" 
{q a ,a'{u)} such that, for each we!!, 

L E«CaM < OO, 

ii. the quadratic form corresponding to {q a . a >} is pos- 
itive definite, 

iii. q aj a = 1, for each a, and hence (by the Schwarz 
inequality) also: |g Q , Q '| < 1 for all pairs {a, a'}. 

An important class of ROSt's is provided by the 
Derrida-Ruelle probability cascade model which is for- 
mulated in rcf. [5] (called there GREM). 

Without additional assumptions, one may associate to 
the points in any ROSt two independent families of cen- 
tered Gaussian variables {rij ia }j=i,2,... and {n a } with co- 
variances (conditioned on the random configuration of 
weights and overlaps) 



from 1 to N, and 



^(Vj.aVj'.a' I q a ,a> ) = 7 S 3,3' /'(&*,<*' 



(7) 



E ( K a K a , | q ata > ) = q a , a >f'(q a , a >) - f{q a ,a>) ■ (8) 

The existence of such processes requires positivc- 
definitcness of the joint covariance, but that is evident 
from the following explicit construction in the case that 
the a's are N- vectors, with q a-a > = . ctj, a'^. 

[N \ - Vra r \ - ~ . . 
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where the second sum is over i\, . . . ,i r -i which range 
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We shall now denote by E(-) the combined average, 
which corresponds to integrating over three sources of 
randomness: the SK random couplings { Jij}, the random 
overlap structure described by the measure n(dw), and 
the Gaussian variables {n a } and 

Guided by the cavity picture, we associate with each 
ROSt the following quantity: 




where a — (ci, . . . , om) 

Our main result is: 
Theorem 1 i. For any finite M, 

P M {0,h) < inf G M {fi,h;n) < Pu(0,h). 



(11) 



(12) 



where the infimum is over random overlap structures 
(ROSt's) andPjj{(3,h) denotes the free energy x(— /?) ob- 
tained through the Parisi "ultrametric" (or "hierarchal") 
ansatz. 

ii. The infinite volume limit oj the free energy satisfies: 
P(0,h) = lim inf G M (0 , h; fj,) . (13) 

Proof These results can be seen as consisting of two 
separate parts: lower and an upper bounds, which are 
derived by different arguments. 

i. The upper bound: the left inequality in eq. (12), 
employs an interpolation argument which is akin to that 
used in the analysis of Guerra [4], but which here is for- 
mulated in broader terms without invoking the ultramet- 
ric ansatz. The second inequality in (12) holds since the 
Parisi calculation represents the restriction of the varia- 
tion to the subset of hierarchal ROSt's. 

To derive the first inequality let us introduce a fam- 
ily of Hamiltonians for a mixed sytem of M spins a — 
(cti, . . . , ctm) and the ROSt variables a, with a parameter 
< t < 1: 



- H M (cr,a;t) = \f\ - t yK M (<j) + y -y «o 

M M 

+ Vt^nj^o-j + h^trj, (14) 

and let 

R M (0,h;t) = ±E(log'^ ?Q '< 1 
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Then 



R M (f3,h;0) = P M (/3,h), (16) 
R M (0,h;l) = G M (J3,h;n), (17) 

and we shall show that -^Rm(P, h; t) > 0. 

We use the following notation for replica averages over 
pairs of spin and ROSt variables. For any X = X(a,a) 
and Y = Y(a, a; a', a'): 

E { t 1} (X) = e! w(a,a;t) X J (18) 

\ a, a / 

E[ 2) (Y) = E I J2 E w(a>,e/;t) Y 



ot.cr a' .a' 



with the "Gibbs weights" 

E 2 t w(a,a;t) = ^e"^^*)/^ ^ e -^(<T,a;t) . 

a,cr 

(19) 



We now have 



t) . (20) 



The term -^Hm{o~, a; t) includes Gaussian variables, and 
one may apply to it the generalized Wick's formula 
(Gaussian integration-by-parts) for correlated Gaussian 
variables, x\,...,x n : 

Av(xiip(xi,...,x n )) = 

= EAv(^)Av( %1 fe ;- I " ) ). (21) 



The result is (after an elementary calculation): 

(jt HM ^ t] ) = t e * 2)( ^ (22) 

with 

ip(a, a; a , a ) = 

= [f(q*,<r') - /(<?<*,«')] ~ {<l<7,a> ~ ?a,a')/' (<?<*,«') • (23) 



As was noted earlier, a particular class of random 
overlap structures is provided by the Derrida-Ruelle 
probability cascade models (GREM) of [5], which arc 
parametrized by a monotone function x : [0, 1] — > [0, 1]. 
These models have two nice features: i. the distribution 
of is invariant, except for a deterministic scaling fac- 
tor, under the multiplication by random factors as in (11) 
(consequently the value of Gm{- ■ ■ , Mx(-)) f° r such ROSt 
does not depend on M), ii. quantities like Gm{- ■ ■ >Mx(-)) 
can be expressed as the boundary values of the solution 
of a certain differential equation, which depends on x(-). 
Evaluated for such models Gm{- ■ ■ >Mx(-)) reproduces the 
Parisi functional for each value of the order parameter 
x(-). The Parisi solution is obtained by optimizing (tak- 
ing the inf) over the order parameter x(-). This relation 
gives rise to the second inequality in (12). 

ii. To prove (13) we need to supplement the first in- 
equality in (12) by an opposite bound. 

Our analysis is streamlined by continuity arguments, 
which are enabled by the following basic estimate (proven 
by two elementary applications of the Jensen inequality). 
Lemma 2 Let Z{H) denote the partition function for a 
system with the Hamiltonian H(a), and let U(a) be, for 
each a, a centered Gaussian variable which is indepen- 
dent of H. Then 



< E log 



Z(H+U) 
Z{H) 



< 



\e{u 2 

2 v 



(25) 



Using the above, it suffices to derive our result for in- 
teractions with the sum over r, in (3), truncated at some 
finite value. 

A convenient tool is provided by the superadditivity 
of Qn = N Pn, which was established in the work of 
Guerra-Toninclli [8] and its extensions [7, 9]. The state- 
ment is that for the systems discussed here (and in fact 
a broader class) for each M, N e N 



Qm+jv(/3, h) > Q M (P, h) + Q N (P, h) 



(26) 



Therefore, 

±R M (J3,h;t) = ^ x (24) 
at 4 

IE| 2) ([/(<?a, CT ') - Z(«a,a')] ~ ~ 9a,«')/' > • 

The last inequality, which is crucial for us, follows from 
the assumed convexity of /. For the SK model, the above 
expression simplifies to E^ ( {q„M' — q a ,a') 2 )- 

Putting the positivity of the derivative together with 
(16) and (17) clearly implies the first bound in (12). 



The superadditivity was used in [8] to establish the exis- 
tence of the limit limAr^oo P N . However, it has a further 
implication based on the following useful fact. 
Lemma 3 For any superadditive sequence {Qn} satis- 
fying (26) the following limits exist and satisfy 

lim Q N /N= lim liminf [Qm+n - Qn]/M . (27) 

N^oo M— >oo TV— »oo 



For our purposes, this yields: 



lim Pn 

N^co 



lim liminf — E flog ^H+E. \ . (28) 
M-,00 N^oo M \ Zn 



We now claim, based on an argument employing the 
cavity picture, that for any M 



lim inf — E ( log %^ 

w^oo M V Z N 



> inf G M ifi,h;n), (29) 
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which would clearly imply (13). The reason for this in- 
equality is that when a block of M spins is added to a 
much larger "reservoir" of TV spins, the change in the free 
energy is exactly in the form of (11) except for correc- 
tions whose total contribution to Gm is of order O(j^). 
(The spin-spin couplings within the smaller block and the 
subleading terms from the change TV i— » (TV + M) in (3).) 
Thus, the larger block of spins acts as a ROSt. 

To see that in detail, let us split the system of M + TV 
spins into a — (a, a), with a — (<7i, . . . , <jm) and a = 
{o~m+i, ■ ■ ■ , (?m+n)- With this notation, the interaction 
decomposes into 

M 

K m +n(v) = K N (a) +^2vj,a<7j +U(a,a) (30) 

where: i. {Kn(o)} consists of the terms of Km+n{&) 
which involve only spins in the larger block, it. the second 
summand includes all the terms which involve exactly 
one spin in the smaller block, and Hi. U consists of the 
remaining terms of Km+n(v), including the spin-spin 
interactions within the smaller block. 

One should note that {K^(a}} ^ {K^(a)} since, as 
a consequence of the addition of the smaller block, the 
terms in {Kjy(a)} are weighted by powers of (TV + M) 
rather than TV, as presented in (3). By the law of addi- 
tion of independent Gaussian variables, {Kn(o>)} (which 
are of higher variance than {Kn(cc)} ) have the same 
distribution as the sum of independent variables 



the limit TV — > oo the SK Gibbs state develops a hierar- 
chal organization. The reasons for such an organization, 
which is equivalently expressed in terms of "ultrametric- 
ity" in the overlaps g CT ,cr', are not a-priori obvious. (A 
step, approaching the issue from a dynamical perspec- 
tive, was taken in ref [10], but this result has yet to be 
extended to the interactive cavity evolution.) Our result 
(12) raises the possibility that perhaps some other orga- 
nizing principles may lead to even lower upper-bounds. 
This reinstates the question whether the ultrametricity 
assumption, which has enabled the calculation of [2], is 
correct in the context of the SK-type models. 

It should be emphasized, however, that the question 
is not whether the SK model exhibits replica symme- 
try breaking at low temperatures. That, as well as 
many other aspects of the accepted picture, are sup- 
ported by both intuition and by rigorous results ([11, 
12, 13, 14, 15]). The question concerns the validity of a 
solution-facilitating ansatz about the hierarchal form of 
the replica symmetry breaking. The interest in this ques- 
tion is enhanced by the fact that this assumption yields 
a computational tool with many other applications [3] . 
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K N (a) 




(31) 



where {n a } are centered Gaussian variables independent 
of Kn(o). Up to factors [1 + O(^)], the covariances of 
{Vj,a} an d {n a } satisfy (7) and (8), respectively, and 



E(U(a, a) 2 ) < C^r . 



Taking 



£ Q := exp 



K N (a) 



N 
i=l 



(32) 



(33) 



we find that (29) follows by directly substituting the 
above into (11) (using (25) and (32)). □ 
Discussion At first glance, the recent result of [4] may be 
read as offering some support to the widely shared belief 
that the Parisi ansatz has indeed provided the solution of 
the SK model. However, we showed here that the Guerra 
bound is part of a broader variational principle in which 
no reference is made to the key assumption of [2] that in 
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